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Graduate attributes and course learning outcomes are an integral part of higher education in Australia. Testing the 
performance of graduates in the workplace with regard to graduate attributes and course learning outcomes is a not a 
common occurrence. This study has road tested the graduate attributes and course learning outcomes of a bachelor 
degree in environmental science via students participating in a work-integrated learning (WIL) placement. 
Comparisons of importance and perceptions of students' skill level between host-supervisors, students and teaching 
staff were made. On the whole teaching staff perceived students' skills as "adequate" while the students and host- 
supervisors perceive students' skill level to approach "proficient". Students on WIL placements appear to meet hosts' 
expectations and there appears to be no significant gap in the curriculum. Road testing graduate attributes and course 
learning outcomes has led to changes in the curriculum. A recommendation is made to define the skill level of a work- 
ready graduate that is quantifiable rather than use terms such as "well-developed". (Asia-Pacific Journal of Cooperative 
Education, 2017,18(1), 1-13) 
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The Australian Qualifications Framework describes the learning outcomes for all levels of post¬ 
secondary school learning from Level 1 (Certificate I) to Level 10 (PhD) (Australian 
Qualifications Framework Council [AQFC], 2013). The graduates of bachelor degrees (Level 
7) "will have broad and coherent knowledge and skills of professional work and/or further 
learning" (AQFC, 2013, p. 13). Qualification learning outcomes require knowledge of the 
discipline, skills and the application of skills and knowledge. As a consequence of a recent 
revision of the Australian Qualification Framework, universities in Australia have 
developed/reviewed course (i.e., degree) learning outcomes that comply with this 
framework. 

The definition and articulation of threshold learning outcomes was commenced in 2010. The 
Australian Teaching and Learning Council, Learning and Teaching Academic Standards 
project facilitated discipline communities to define and articulate learning and teaching 
academic standards statements (i.e., threshold learning outcomes). The statements defined 
the minimum learning outcomes a graduate must achieve for discipline-specific knowledge, 
discipline-specific skills, generic skills and discipline-specific capabilities (Australian 
Learning and Teaching Council, 2010). Harris (2009) summarized initiatives including the 
tuning process (Europe) that identified threshold-level learning outcomes and subject 
benchmark statements (UK). In the UK, learning outcomes form part of the national quality 
assurance framework (Harris, 2009). Threshold learning outcomes have been published for 
28 disciplines in Australia 2 (Freeman & Ewan, 2014); the future of threshold learning 
outcomes is unclear in relation to the Tertiary Education Quality and Standards Agency but 
institutions are expected to review existing course learning outcomes and align them with 
threshold learning outcomes of the discipline. Threshold learning outcomes were recently 
released for environmental and sustainability (Phelan et al., 2015). 


1 Corresponding author: Michael Whelan, Michael.Whelan@scu.edu.au 

2 See http://www.olt. gov.au/search/apachesolr search/Academic%20standards 
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Development of sound learning outcomes that are clearly communicated, observable, 
demonstrable and measurable is the aim of most institutions (Baume, 2009). One strategy to 
achieve sound learning outcomes is to align intended learning outcomes with experiences 
and assessment (Biggs & Tang, 2011). The Australian Qualification Framework requires 
learning outcomes that lead to "knowledge and skills of professional work" (AQFC, 2013, p. 
13) and industry has the expectation that graduates are work ready (Australian Industry 
Group, 2015). However, the employability of graduates, which is difficult to measure, is 
embedded in graduate attributes and qualification learning outcomes (Knight & Yorke, 2003; 
Oliver, 2011). In a review of "assuring graduate outcomes" Oliver (2011) presented a number 
of strategies developed by Australian universities to achieve graduate attributes. While 
Jackson (2010) was clear in identifying the attributes of employability, measuring the 
employability of graduates was difficult. The employability of graduates with potential 
employers has been assessed for accountants (Oliver, Whelan, Hunt, & Hammer, 2011) and 
health science and humanities (Ferns, 2012). 

Jackson (2010) suggested conducting an empirical study of undergraduates who participate 
in internships towards the end of their degree program to measure employability. Smith, 
Ferns and Russell (2014) quantified the work readiness of graduates using 35 attributes to 
calculate a single index. They determined that there was a positive relationship between the 
quality of the WIL experience and work-readiness of graduate. An employer satisfaction 
survey (Oliver, Freeman, Young, Yu, & Verma, 2014) evaluated skills. However, there was 
no attempt to quantify the skill level required to meet an employability standard. There is 
little research about road testing graduate learning outcomes when graduates enter the 
workforce. Although, Yung (2010) has compared the graduate learning outcomes from Hong 
Kong University with the performance assessed by employers. 

Work-integrated learning (WIL) has for many years been considered an authentic experience 
that helps develop graduates' employability skills and has great benefit to students (Patrick, 
Peach, & Pocknee, 2008; Smith et al., 2014). Internships are the most common form of WIL 
(PhillipsKPA, 2014). Ninety percent of industry participating in WIL identified access to 
work-ready graduates as a benefit of participation in WIL (PhillipsKPA, 2014). While WIL is 
a requirement for disciplines that require accreditation (e.g., health professionals, teachers 
and engineers) it is not a requirement in environmental or science disciplines. A recent 
review of WIL in science, technology, engineering and mathematics (STEM) in Australian 
universities highlighted the lack of participation in WIL by STEM students in WIL (Edwards, 
Perkins, Pearce, & Hong, 2015) with only one in seven science graduates having a WIL 
experience. If results from previous studies (Patrick et al., 2008; PhillipsKPA, 2014; Smith et 
al., 2014) and assumptions that underpin the "National WIL Strategy" (Universities 
Australia, Australian Chamber of Commerce and Industry, Australian Industry Group, 
Business Council of Australia, & Australian Collaboration Education Network, 2015) can be 
replicated in STEM disciplines then there is a potential to improve the employability of 
STEM graduates by increasing participation in WIL. The Chief Scientist of Australia has 
recently funded a project to increase WIL participation in science (Australian Council of 
Deans of Science, 2015). In addition, WIL provides an opportunity to measure the attainment 
of graduate attributes and graduate learning outcomes in an authentic, quantifiable way 
(Jackson, 2010). 

One of the goals of WIL is to deepen classroom conceptions, apply skills and make the 
curriculum more meaningful to students (Orrell, 2011). One way to make the curriculum 
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more meaningful is to review the curriculum in response to feedback of prospective 
employers of graduates. WIL is unique in the higher education landscape in that it provides 
the opportunity for 360° feedback between the university and industry (Patrick et al., 2008). 
Amongst the benefits identified by industry participating in WIL, 45% of respondents 
identified the input to the curriculum as a benefit (PhillipsKPA, 2014). On the other hand, 
only one respondent of 60 identified providing feedback on curriculum as a benefit 
(Atkinson, Misko, & Stanwick, 2015) and curriculum development was not listed as a benefit 
in a scoping study (Australian Workforce and Productivity Agency, 2015). Although the 
graduate learning outcomes are stated for a degree, the way students engage in WIL to meet 
those graduate learning outcomes will vary depending on their placement (Yorke, 2011). 

The aim of the present study was to road test course learning outcomes of a degree in the 
environmental discipline using feedback from hosts, students and teaching staff. Specific 
objectives were as follows: 

• Design a survey instrument that would quantify the importance and student skill 
level of graduate attributes and course learning outcomes. 

• Compare the perceptions of importance and student skill level of graduate 
attributes and course learning outcomes between host-supervisors, students and 
teaching staff. 

• Evaluate how road testing course learning outcomes can be used to review the 
curriculum. 

The aims of the present study are relevant to two of the recommendations presented in the 
"Good Practice Report: Work-Integrated Learning" (Orrell, 2011, p. 20): 

• WIL programs are integrated into the curriculum so that they have clear educational 
expectations, and are a vehicle for integrating theory and practice learning: 

• Evidence from a variety of sources is used to monitor, evaluate and improve the 
effectiveness of diverse WIL program arrangements. 

METHODOLOGY 

The methodology was modeled on a study that used the Graduate Employability Indicator 
Survey developed by Oliver, (2011) in health science and humanities disciplines (Oliver as 
cited by Ferns, 2012). The present study compared the perceptions of importance and 
perceived student skill level of course learning outcomes and graduate attributes. The three 
major stakeholders in WIL were included (host-supervisors, students and teaching staff). 
The sample was drawn from the 2014 cohort of students enrolled in an internship unit, the 
supervising staff in the host organizations and teaching staff who were responsible for 
teaching the degree. The internship unit requires a placement equivalent to eight weeks of 
full-time work. Students were not paid for their internship. The data were collected using 
Qualtrics on-line survey (students and hosts) and a printed survey (teaching staff). Approval 
was granted by the Human Research Ethics Committee, Southern Cross University (ECN-13- 
192). Students completed the survey after their placement. Hosts were invited to participate 
after they had completed their evaluation of the student's performance. Teaching staff 
completed the survey at school meetings. Only quantitative data were collected in the 
present study. The rating system for perceived skill level was based on the generic 
descriptions of stages of performance developed by the Department of Industry, Innovation, 
Climate Change, Science, Research and Tertiary Education and Department of Education, 
Employment and Workplace Relations [DIICCSRTE & DEETWR], (2013). 
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Table 1: Graduate attributes (numbered) and course learning outcomes (nested with letters) 
in the questionnaires for students, students' host-supervisor and teaching staff. 



Full graduate attribute and course learning outcome for BEnvSc 

Abbreviated graduate 
attribute and course learning 
outcome 

1 . 

Intellectual rigor: A commitment to excellence in all scholarly 
and intellectual activities, including critical judgement. 

Intellectual rigor 

a) 

Demonstrate skills of critical analysis, and application of scientific 
methods in environmental science and management 

Critical analysis & application 

b) 

Make decisions and exercise informed judgement in relation to 
environmental science and management 

Decision making 

2. 

Creativity: An ability to develop creative and effective 
responses to intellectual, professional and social challenges. 

Creativity 

a) 

Demonstrate imagination, initiative and enterprise in problem¬ 
solving 

Imagination in problem 
solving 

b) 

Respond creatively to intellectual, professional, environmental 
and social challenges 

Respond creatively 

3. 

Ethical practice: A commitment to sustainability and high 
ethical standards in social and professional practices. 

Ethical practice 

a) 

Evaluate issues with reference to sound ethical frameworks and 
sustainability 

Sound ethical framework 

b) 

Demonstrate well developed judgement on principles of social 
justice and professional standards 

Developed standards 

4. 

Knowledge of a discipline: Command of a discipline to enable 
a smooth transition and contribution to professional and 
community settings. 

Discipline knowledge 

a) 

Demonstrate broad and coherent knowledge of environmental 
science and management 

Broad coherent knowledge 

b) 

Apply disciplinary knowledge and skills in professional and 
community settings 

Apply skills 

c) 

Demonstrate in-depth knowledge in one or more disciplines, or 
areas of practice 

In-depth knowledge 

5. 

Lifelong learning: The ability to be responsive to change, to be 
inquiring and reflective in practice, through information 
literacy and autonomous, self-managed learning. 

Lifelong learning 

a) 

Demonstrate cognitive and technical skills in self-managed 
learning 

Self-managed learning 

b) 

Critically reflect on practice 

Critically reflect 

c) 

Demonstrate information literacy skills 

Information literacy 

6. 

Communication and social skills: The ability to communicate 
and collaborate with individuals, and within teams, in 
professional and community settings. 

Comm. & social skills 

a) 

Communicate clearly and coherently knowledge and ideas in 
environmental science and management contexts 

Communicate clearly 

b) 

Collaborate effectively on personal, scholarly, and professional 
terms 

Collaborate effectively 

7. 

Cultural competence: An ability to engage with diverse cultural 
and Indigenous perspectives in both global and local settings. 

Cultural competence 

a) 

Demonstrate awareness and respect for cultural diversity and the 
relationship between people and their environment 

Cultural awareness 
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Hosts were asked to describe their organization (i.e., local, state or federal government, 
government authority, private company or not for profit organization) and how well their 
activities aligned with the majors in the degree (coastal management, natural resource 
management, fisheries and aquaculture or marine science). Students were asked how many 
units/subjects/papers, of the 24 required to fulfil the requirements of the degree, they had 
completed (12-15 units, 16-19 units, 20-23 units, more than 23 units). 

All stakeholder groups were asked the level of importance (Meaning not clear, Not 
applicable (hosts only), "0 Unimportant", 1, 2, 3, 4, "5 Great Importance") they place on the 
graduate attributes and course learning outcomes (Table 1). They were then asked to indicate 
the skill level (Meaning unclear, Not applicable, 0 No skills, 1 Low skill level, 2 Basic skills, 3 
Adequate skills, 4 Proficient, 5 Professional) the student had achieved for the graduate 
attributes and course learning outcomes. Students were asked to rate their own skill level. 

All statistical analysis was carried out using SPSS Version 22. One-way analysis of variance 
was used to compare means and Tukey HSD was used as a post hoc test. Pearson product 
moment correlation coefficients were calculated to highlight redundancy in the data. 
Euclidian distance was used to quantify the multidimensional distance between hosts, 
students and teaching staff. 

RESULTS 

The number of respondents (who agreed to their responses being used in this research) in 
each of the categories were as follows (hosts - 24, students - 39, teaching staff - 8). A 
summary of the responses of students and hosts are presented in Table 2. 


TABLE 2: Broad categories of type and discipline of host organizations and number of units 
completed by students (24 required for the degree). 


Host: organization 

n 

Host: discipline 

n 

Student: units 

n 

Local government 

5 

Coastal management 

i 

12-15 

0 

State government 

12 

Natural resource management 

16 

16-19 

5 

Federal government 

1 

Fisheries and aquaculture 

2 

20-23 

23 

Private 

4 

Marine science 

2 

>23 

10 

Not for profit 

2 

Other 

3 




As Figure 1 illustrates teaching staff placed a significantly (p<0.05) lower level of importance 
on communication and social skills (4.13) than hosts (4.68). Lifelong learning was rated of 
high importance by all stakeholders. Although teaching staff and students appeared to place 
greater importance on ethical practice than hosts, the differences were not significant. There 
were similar responses to creativity by all stakeholders. Students appeared to place greater 
importance on cultural competence than hosts and teaching staff but only the difference 
between students (4.50) and teaching staff (3.63) was significant. While students and 
teaching staff both placed great importance on discipline knowledge, hosts scored it quite 
low. The difference between students (4.29) and hosts (3.42) was significant (p<0.05). 
Intellectual rigor was given the lowest mean importance of all graduate attributes by the 
hosts. However, teaching staff placed greatest importance on intellectual rigor and mean 
scores of teaching staff (4.50) and students (4.36) were significantly different (p< 0.05) to hosts 
(3.38) (Figure 1). 
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FIGURE 1: Comparison of the mean importance of graduate attributes (refer to Table 1) 
between hosts (n=24), students (n=39) and teaching staff (n=8). Columns are ranked on the 
level of importance placed on the graduate attribute by the host supervisors. 

A comparison between hosts, students and teaching staff of the perceived student skill level 
of graduate attributes is illustrated in Figure 2. The columns in Figure 2 are presented in the 
same order as Figure 1 (i.e., the host's most important graduate attribute on the left and the 
least important graduate attribute on the right). In general teaching staff perceived students' 
skill level as adequate (3.2 on the 5 point scale), while hosts and students perceived student' 
skill level as proficient (3.8 and 3.9, respectively, on the 5 point scale) (Figure 2). 

Statistical analysis identified significant (p<0.05) differences in perceived skill level in the 
following graduate attribute comparisons. 

• Lifelong learning: between teaching staff (3.00) and students and hosts (3.94 and 
4.11, respectively) 

• Ethical practice: between teaching staff (3.00) and students and hosts (3.94 and 4.06, 
respectively) 

• Creativity between: the teaching staff (3.00) and students (3.83) 

• Cultural competency: between teaching staff (2.75) and students and hosts (3.83 and 
4.08, respectively) 
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FIGURE 2: Comparison of the perceived skill level of students graduate attributes (refer to 
Table 1) between hosts n=19, students n=37 and teaching staff n=8. Columns are ranked on 
the level of importance place on the graduate attribute by the host supervisors. 

The importance placed on different course learning outcomes by different stakeholders is 
presented in Table 3. The emphasis on communication and social skills is consistent with the 
results of graduate attribute importance ratings by hosts. 

TABLE 3: Comparison of the importance of course learning outcomes (refer to Table 1) 
between hosts n=22, students n=37 and teaching staff n=8. 


Course learning objective 

Host 

Student 

Teaching staff 

Critical analysis & application 

4.09 

4.59 

4.63 

Decision making 

4.00 

4.69 

4.25 

Imagination in problem solving 

4.00 

4.55 

4.13 

Respond creatively 

3.86 

4.32 

4.63 

Sound ethical framework 

4.24 

4.47 

4.13 

Developed standards 

4.00 

4.46* 

3.63* 

Broad coherent knowledge 

4.05* 

4.50 

4.75* 

Apply skills 

3.82 

4.45 

4.25 

In-depth knowledge 

3.62 

4.24 

4.13 

Self-managed learning 

3.76 

4.21 

3.75 

Critically reflect 

3.76 

4.32 

3.75 

Information literacy 

3.71 

4.11 

4.25 

Communicate clearly 

4.36 

4.70 

4.25 

Collaborate effectively 

4.33 

4.37 

3.88 

Cultural awareness 

4.05 

4.58* 

3.88* 


’‘'Significant difference (p<0.05). 
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Teaching staff, overall, perceived that students' skill level was lower than hosts' and 
students' perceptions (Table 4). Teaching staff perceived student skill level as adequate (3.2 
on the 5 point scale) while hosts and students perceived students' skill level as proficient 
(both 3.8 on the 5 point scale). Teaching staff perceptions of students' skill level were 
significantly lower (p<0.05) than hosts' perception for five course learning outcomes and they 
were significantly lower than students' perception of their own skill level for eight course 
learning outcomes. In contrast, students' and hosts' perceptions of students' skills were 
similar (Table 4) and there were no significant differences. 


TABLE 4: Comparison of the perceived skill level of course learning outcomes (refer to Table 
1) between hosts n=18 and students n=37 and teaching staff n=8. 


Course learning objective 

Host 

Student 

Teaching staff 

Critical analysis & application 

3.61 

3.84 

3.13 

Decision making 

3.53 

3.97* 

3.25* 

Imagination in problem solving 

3.70 

4.00* 

3.13* 

Respond creatively 

3.75 

3.97* 

3.13* 

Sound ethical framework 

3.78a 

3.89a 

3.00 b 

Developed standards 

3.65 a 

3.66 a 

2.71 b 

Broad coherent knowledge 

3.71 

3.62 

3.75 

Apply skills 

3.78 

3.49 

3.38 

In-depth knowledge 

3.72 

3.38 

3.00 

Self-managed learning 

3.88 

3.92* 

3.25* 

Critically reflect 

3.93 a 

3.94 a 

2.50 b 

Information literacy 

4.06 

3.80 

3.75 

Communicate clearly 

4.00 

3.76 

3.75 

Collaborate effectively 

4.13* 

3.81 

3.25* 

Cultural awareness 

4.15 a 

4.08 a 

2.75 b 


Where: * used where one difference was detected (p<0.05) and letters used where two differences were detected. Means with the same 


letter are not significantly different (p<0.05). 


Euclidian distance highlighted the contrast between the teaching staff's perception of 
students skills and perceptions of skills by hosts and the students themselves (Table 5). 
Figure 3 illustrates the clustering of the three groups. The relationship between importance 
and perceived skill level was positive (r=0.58) and significant (p<0.05) for teaching staff and 
not significant for students (r=0.22) and hosts (r=0.16). 
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TABLE 5: Euclidian distance between groups (hosts, teaching staff and students). 


Comparison 

Graduate 

attribute 
perceived 
skill level 

Course learning 
outcome 
perceived skill 
level 

Graduate 

attribute 

Importance 

Course 

learning 

outcome 

Importance 

Mean 

Distance 

Teaching staff vs host 

3.95 

2.92 

2.36 

1.61 

2.71 

Teaching staff vs student 

4.19 

3.00 

1.24 

1.71 

2.53 

Host vs student 

0.26 

0.92 

2.98 

1.89 

1.51 



0 


0 



O Oiejchrng staff 

□Host 
• Student 


1 2 3 4 5 

Mean Importance of graduate attributes and course learning outcomes 


FIGURE 5: Clustering of groups when importance is plotted against perceived skill level. 


DISCUSSION 
Road Test 

Road testing revealed that overall, graduates of the degree were meeting the expectations of 
potential employers. However, this statement is based on a few assumptions. First, that the 
students on their placement were equivalent to a new graduate. Jackson (2010) considered 
that students completing a WIL placement close to the end of their degree could be used to 
measure employability. Given that most of the students had completed 20+ units of the 24- 
unit degree (Table 2) they could be considered equivalent to new graduates and often the 
internship is the final unit for their degree. Secondly, the hosts who worked alongside the 
students for eight weeks were in a position to evaluate their skills, and because the internship 
program has been running since 2002 the hosts-supervisors were frequently graduates of the 
degree and often interns themselves when they were students. Finally, it assumes that the 
graduate attributes and course learning outcomes cover all of the skill requirements of a new 
graduate. 

Road Test: Importance 

On the whole, there were few differences between hosts, students and teaching staff with 
regard to importance of graduate attributes and course learning outcomes. The host 
response revealed that graduate attributes and course learning outcomes were rated between 
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3.4 and 4.7 on a 5 point scale of importance (Figure 1 and Table 3). The graduate attributes 
Intellectual rigor (3.4) was considered the least important by the hosts but the course learning 
outcomes that grouped within the graduate attributes had quite high importance scores 
(Critical analysis & application - 4.1 and Decision making - 4.0). A study that compared the 
importance teaching staff and students placed on transferable skills embedded in science 
TLOs also found close agreement with 3 of 5 learning outcomes, while teamwork and ethical 
framework were considered more important by students (Matthews & Mercer-Mapstone, 
2016). 

Road Test: Perceived Skill Level 

Overall, the hosts and students were satisfied with the students' skill level. The lowest 
perceived skill level for a graduate attributes (creativity) by the hosts was 3.6 (between 
adequate and proficient) and the lowest value for a course learning outcome was 3.5 for 
decision making (make decisions and exercise informed judgment in relation to 
environmental science and management). However, the hosts were not asked "What skill 
level do you expect of a new graduate?". 

Australian Qualification Framework Level 7 (bachelor degree) descriptions use the term 
"well-developed" to define the required skill level (e.g., graduates at this level will have well- 
developed cognitive, technical and communication skills). While at Level 8 the skill level is 
defined as 'advanced' and Level 9 and 10 the skill level is defined as 'expert' (AQFC, 2013, p. 
13 - 14). To road test graduate skill level of a degree against Australian Qualification 
Framework Level 7 a definition of "well-developed" is required. On the other hand, expert is 
defined as "Expert knowledge and/or skills are the highest level of skills underpinned by 
extensive knowledge or ability based on research, experience or occupation in a particular 
area of study" (AQFC, 2013, p. 24). The generic descriptions of stages of performance 
developed by the DIICCSRTE and DEETWR (2013) are clearly defined. In future editions of 
Australian Qualification Framework it would be useful to define the skill level required. 

An advantage of the rubric developed by DIICCSRTE and DEETWR (2013) is that the 
descriptors are focused on behavior. For example, an expert "operates fluidly, intuitively 
and flexibly in highly complex situations", while a novice "is highly reliant on explicit 'rules' 
(e.g., instructions, processes, procedures, models), guidance and support and priorities 
determined by others" (DIICCSRTE & DEETWR, 2013, p. 7). 

Road Test: Teaching Staff Perceptions 

Teaching staff, on the whole, had less confidence in the students' skill level than hosts and 
students (Table 4). Ferns (2012) found similar results in her study of graduate, employer 
and teaching staff perceptions of employability. Teaching staff tended to rate the skill level 
of a course learning outcome if they placed greater importance on it (Figure 3). Teaching 
staff perceived the skill level high in communicate clearly, broad coherent knowledge and 
information literacy (all 3.8). These course learning outcomes could be considered easily 
assessed using traditional assessments such as exams, reports and essays. Teaching staff 
perceived the skill level of students low in cultural awareness, developed standards and 
critically reflect (2.8, 2.7, 2.5 respectively). These learning objectives may require more 
sophisticated assessment techniques. 

The perceptions of course learning outcome performance of the students and hosts was 
very similar (Table 4) and this was reflected in the Euclidian distance (0.92) in comparison 
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of the distance between hosts and teaching staff (2.92). The alignment between hosts and 
students could be a consequence of good supervision. If employers are praising good 
performance and encouraging improvement when performance is lacking then students 
will have insight into their skill level. A frequent comment in students' journals is that they 
are praised for good work and realize when they need to improve their performance. As a 
consequence of working with colleagues and being judged on their behavior rather than 
their performance in assessment items the interns improve their self-efficacy (Thompson, 
Bates, & Bates, 2016). 

The difference in perception is an issue that could be explored in greater detail and a 
qualitative approach would be required to explore the expectations of teaching staff. The 
author has coordinated the internship unit for the past seven years and the host-supervisors' 
evaluation of students has always been high. Interestingly, students' with poor academic 
grades often do very well in their placement. In one case, a student who was considered a 
poor performer by an academic, teaching in a particular discipline, was employed by a host 
in the same discipline after her internship. A student's academic record is not considered as 
part of the selection criteria in the internship although students do highlight good results in 
their resumes. In addition, there are usually more positions available than students seeking 
placements and so there is little competition. In 2015, of the 40 students seeking a placement 
there was only one position that was competitive. 

Road Test: Reviewing the Curriculum 

There is a need for WIL programs to be integrated into the curriculum so that they are a 
vehicle for integrating theory and practice (Orrell, 2011). Feedback from WIL can be used to 
improve work-readiness (Oliver, 2011). The results of this study will provide an evidence- 
based focus for curriculum development in the degree. Capabilities identified by employers 
of STEM graduates (1- active learning, 2 - critical thinking, 3 - complex problem-solving and 
4 - creative problem-solving (Deloitte Access Economics Pty Ltd & Office of the Chief 
Scientist, 2014) were also highlighted in the present study. The level of importance hosts 
placed on communication and teamwork provided the evidence that led to a review of 
curriculum focused on teamwork and communication. Teamwork was introduced as a topic 
in a first year core unit as a consequence. Matthews and Mercer-Mapstone (2016) reported 
that students placed greater importance on teamwork than teaching staff but both groups 
considered teamwork was adequately included in the curriculum and assessed. 

The Survey Instrument 

Using the generic descriptions of stages of performance developed by the DIICCSRTE and 
DEETWR (2013) will make it easier to benchmark this study with others. Asking hosts to 
define the skill level they require of a new graduate would have strengthened the survey 
instrument. In addition, asking the hosts if there were skills not included in the survey, may 
have revealed gaps in the curriculum. It is recommended that the rubric developed by 
DIICCSRTE and DEETWR (2013) is incorporated into future studies of work-readiness of 
graduates. 

Improving the WIL Program 

There is a need to use evidence from a variety of sources to monitor, evaluate and improve 
the effectiveness of WIL programs (Orrell, 2011). The results of this study provided the 
evidence to improve the evaluation of students' performance at the end of their placement. 
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A scale from 1 (poor) to 9 (excellent) to rate students' performance in a number of attributes 
was replaced by the generic descriptors (rubric) developed by the DIICCSRTE and DEETWR 
(2013). 

CONCLUSION 

It is clear from the assessment of students' skills by the host-supervisors that students are 
able to perform at an adequate or proficient level. The study has shown that it is possible to 
use feedback from an internship placement to improve the curriculum. A recommendation 
from the study is that there needs to be a clearer definition of the skill level required of a 
graduate from a bachelors degree. The term 'well-developed' isn't adequate. Integration of 
the rubric developed by DIICCSRTE and DEETWR (2013) into the Australian Qualification 
Framework would make the system more robust. A second recommendation is that 
employers are asked to define the skill level of a work ready graduate using a well-defined 
rubric. 
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